Distributed file systems offer many compelling advantages in establishing high performance computing environments. One example is the ability to easily expand, even at large scale. An example distributed file system is one that is distributed across multiple nodes in a cluster of nodes. An individual node can encompass a set of storage drives capable of storing data accessible by clients of the clusters of nodes. In some distributed file systems, files or objects can striped across different physical storage devices in a single node or across multiple nodes of a cluster. With multiple clients capable of interacting and requesting operations on files to multiple nodes, many times there are operations that are executed by multiple threads, processes and applications distributed across one or more nodes. For example, more than one thread or process may demand to concurrently write data to the same file in a file system.
In a distributed file system, such as a cluster of nodes, file system operations can be viewed as multi-layered. The first layer decodes what the operation is trying to accomplish, including assessing which nodes, including the node processing the operation, among the cluster of nodes are storing data that the operation is dependent on. As the operation progresses, a journal can be used as a means to provide more guarantees for requested operations, by first having the operations logged into a journal, (e.g., an in-memory storage space such as a cache or a buffer cache), and then later committing the operations logged in the journal to stable disk storage.
The journal can provide a reliable write-back cache for filesystem disk updates. It can remember the disk updates and re-write them in the event of a power failure or a write failure. This can aid in providing file system consistency in the event of a crash. A bad or corrupted journal can result in data loss, which can require the node containing the bad journal to undergo a rebuild before offering normal functionality. One way to prevent the corruption or loss of a journal is to mirror the journal in multiple locations. If one copy of the journal is corrupted, a mirrored copy of the journal can be used to avoid a lengthy rebuild of the node. However, as the journal is dynamically updated during filesystem operation, it is important to establish a cadence of how and when copies of the journal are mirrored along with establishing internal guarantees for which mirrored copies of the journal are valid.